Skip to content

Fix dtype mismatch in resolution analyses#148

Merged
brunomsaraiva merged 2 commits into
HenriquesLab:mainfrom
alxndrkalinin:fix_decorr_dtype
Nov 27, 2025
Merged

Fix dtype mismatch in resolution analyses#148
brunomsaraiva merged 2 commits into
HenriquesLab:mainfrom
alxndrkalinin:fix_decorr_dtype

Conversation

@alxndrkalinin
Copy link
Copy Markdown
Contributor

@alxndrkalinin alxndrkalinin commented Nov 26, 2025

There is an issue in resolution analyses (decorrelation and FRC), where Cython cdef complex[:, :] defaults to double complex (complex128), but np.fft.fft2() on float32 input returns complex64.

Actually, current tests fail on main, e.g.

pytest tests/test_decorr_analysis.py
pytest tests/test_package_level_calls.py::test_package_decorr
pytest tests/test_package_level_calls.py::test_package_frc

each raise

ValueError: Buffer dtype mismatch, expected 'double complex' but got 'complex float'

Explicitly specifying float complex solves the issue and all three tests pass.

@brunomsaraiva
Copy link
Copy Markdown
Collaborator

brunomsaraiva commented Nov 26, 2025

Hi @alxndrkalinin,

Thanks for the finding and PR, I'll just wait for the CLI to finish running and will then aprove the PR merge and prepare a new NanoPyx release with the fix.

Best,
Bruno

@brunomsaraiva
Copy link
Copy Markdown
Collaborator

brunomsaraiva commented Nov 26, 2025 •
edited
Loading

Hi again @alxndrkalinin,

Could you give me some details about which python version you're running and the cython and numpy version you have installed in your env?
Both the CI testing and local testing for me are giving the errors when defining them as float complex.
Conversely, current version does have any errors on the CI or locally for me.

Best,
Bruno

@alxndrkalinin
Copy link
Copy Markdown
Contributor Author

alxndrkalinin commented Nov 26, 2025

Hi @brunomsaraiva,

Thanks for checking. Turns out, I had two numpy dist-info directories (1.26.4 and 2.3.4). pkg_resources was seeing the 1.26.4 metadata even though Python was actually using 2.3.4 - so version check passed and allowed running tests, while I was actually using numpy 2.x.

I tested to confirm:

code version numpy 1.26 numpy 2.x
cdef complex ✅ Passes ❌ Fails
cdef float complex ❌ Fails ✅ Passes

Not sure what your plans are for numpy 2.x support, but one option to support both is by explicitly casting FFT output to complex64:

  numpy 1.x:  float32 → fft2 → complex128 → .astype(complex64) → matches float complex ✓
  numpy 2.x:  float32 → fft2 → complex64  → .astype(complex64) → matches float complex ✓ (no-op)

This makes tests pass with numpy either <2 and 2.x, so I committed this change here. Let me know what you think.

@brunomsaraiva
Copy link
Copy Markdown
Collaborator

brunomsaraiva commented Nov 27, 2025

Hi @alxndrkalinin,

Thanks for the fix for both numpy versions. Letting CI tests ran and if everything works as expected will merge the PR.

Regarding numpy > 2 support, it is something we are aiming to support and will work on it once we find some time to fully test numpy>2 integration across all methods.

@brunomsaraiva brunomsaraiva merged commit 6622fda into HenriquesLab:main Nov 27, 2025
1 check passed
@brunomsaraiva brunomsaraiva mentioned this pull request Nov 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@alxndrkalinin @brunomsaraiva